Section: New Results

Person Re-Identification using Pose-Driven Body Parts

Participants : Behzad Mirmahboub, Furqan Khan, François Brémond.

keywords: appearance-based person re-identification, pose estimation, human body parts, mask.

Introduction

Person re-identification is the problem of recognizing persons between several non-overlapped cameras. The main assumption is that the persons don't change their clothings between cameras. General approach is to extract discriminating color and texture features form images and calculate their distances as a measure of similarity. Most of the works consider whole body to extract descriptors. However, human body may be occluded or seen from different views that prevents correct matching between persons. We propose to use a reliable pose estimation algorithm to extract meaningful body parts and extract descriptor from each part separately.

Body Parts

“OpenPose” [52] is a state-of-the-art pose estimation algorithm that detects 15 body joints as shown in Fig. 17 (a). An example of pose estimation result on MARS dataset [142] is shown in Fig. 17 (b). We used joint positions to define 12 body parts as shown in Fig. 17 (c). Our idea is to extract image descriptor from each part and calculate their distances separately. Distance between two images can be computed by weighted average of all distances between body parts.

LOMO [90] is a famous descriptor for person re-identification that divides each image into horizontal stripes and finds the maximum bins of color and texture histograms in each stripe. We modified this code to use it on body parts.

Body Mask

Another challenge in person re-identification is different backgrounds between cameras. Background usually adds unnecessary information to descriptors that is not related to the person in the image, resulting in the mismatch between persons. We used the results of pose estimation algorithm to find a mask for whole body as can be seen in Fig. 18. Since the masked image has many zero-value pixels that are not related to the persons, we modified LOMO descriptor to remove the effect of those zero pixels.

Conclusion

Preliminary experiments show some potentials of using pose estimation for ReID, but not as accurate as global signature. One shortcoming of our work may be that we relied on LOMO descriptor that is essentially designed for the whole image. Suitable descriptor such as deep features [130] should be designed for body parts. In case of proper descriptor, part-based re-identification is promising to cope with the problem of pose and view point variations. This work can also be extended to detect mid-level features or attributes [86] (such as gender, long hair, jeans, t-shirt etc.) that are more reliable than low-level descriptors (such as gradients and histogram).